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OVERVIEW 


orkforce development organizations often find it challenging to assess how former program 

participants are faring in the labor market, since they need to rely on participants’ willingness 

to report and verify their job placements after they either leave or complete their programs. 
The 2013 Unemployment Insurance Data Sharing Bill (S5773A) amended the New York State Labor 
Law to make it easier for government agencies to obtain state unemployment insurance (UI) wage data 
for program monitoring, improvement, and evaluation purposes. In 2016, the Change Capital Fund 
(CCF) and the New York City Mayor’s Office for Economic Opportunity (NYC Opportunity) identi- 
fied the law as a chance to invest in a demonstration with four community development organizations 
that were already CCF grantees, aiming to expand their ability to collect and use data to improve their 
programs in coordination with the city government. 


This report describes that two-year demonstration and is meant to serve as a guide for other New York 
municipalities and community organizations that may consider requesting access to state UI wage 
data. It illustrates some of the challenges and opportunities involved in accessing UI data on program 
participants and offers some practical lessons for organizations in New York State. 


KEY LESSONS 

The contract development process with the New York State Department of Labor (NYSDOL) took 
over a year, during which MDRC served as the data intermediary and provided technical assistance 
to the community organizations. Once the agreement was executed, MDRC received, processed, and 
analyzed the data, sharing the findings with each organization. Following are some of the important 
lessons that emerged from the demonstration: 


¢ Contract structure is important in securing data access. Under the 2013 law, NYSDOL can share 
either identifiable (confidential) data or data in deidentified or aggregate form. In order to qualify 
for access to identifiable data, community organizations must be in a direct contractual relationship 
with a federal, state, or city agency; for this reason, NYSDOL did not grant access to identifiable data 
to the community organizations in the demonstration. Deidentified or aggregate data are easier to 
obtain, so organizations may qualify for data access without an agency partner. 


¢ Identifiable data are preferred over deidentified or aggregate data, but aggregate data are still 
informative and valuable. All the CCF grantees would have preferred to receive identifiable data 
so that they could use them to improve services for specific individuals and have the flexibility for 
additional analyses as other questions arose. While aggregate results could not address analysis 
questions as completely as identifiable data could, the grantees still found the data useful for un- 
derstanding employment trends and earnings averages across key subgroups. 


¢ Preparation is crucial to benefiting from the data access. Community organizations should make 
sure that the participant information in their databases is complete and accurate, that staff members 
have the time and ability to make good use of the data, and that analysis questions are thoughtfully 
prioritized before investing in a data acquisition effort with NYSDOL, to maximize the value of UI 
wage data access. 


° Wage data can guide program improvements. Carefully planned analyses with UI wage data to track 
employment and earnings for certain groups of participants can inform program improvements. For 
example, organizations could examine earnings data to assess whether participants without high 
school diplomas need different kinds of employment services than do more educated participants. 
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Introduction 


common challenge for workforce development organizations is tracking employment and 

earnings outcomes for program participants who have either left or completed their job-related 

training. Ifa participant is no longer actively engaged in the program, the organization often 
has no practical way of knowing whether the person was able to find and maintain a job. To allow 
organizations to obtain this information and use it to improve their programs, a recent New York State 
law expands access to identifiable, individual-level employment data for qualifying organizations. 


The New York State Labor Law was amended under the 2013 Unemployment Insurance Data Sharing 
Bill (S5773A) to make it easier for federal, state, and local government agencies, or their agents or 
contractors, to obtain state unemployment insurance (UI) wage and employment data for program 
monitoring, improvement, and evaluation purposes." In the past, if an organization wanted to know 
the UI earnings of a particular individual it served, it could obtain that type of information (called 
identifiable, or confidential, UI data) only by collecting individual signed and notarized authori- 
zations that met the informed consent requirements of the New York State Department of Labor 
(NYSDOL). The 2013 amendment waives the written authorization requirement for “qualified entities 
for certain authorized purposes.” See Box 1 for more details about the law. 


In 2016, the Change Capital Fund (CCF),? a New York City-based collaborative of foundations, 
corporations, and intermediaries dedicated to economic mobility, and the New York City Mayor’s 
Office for Economic Opportunity (NYC Opportunity) identified the law as a chance to build on 
their investment in four community development corporations and expand those organizations’ 
ability to use data to improve their programs in coordination with the city government. CCF and 
NYC Opportunity hoped the project, if successful, would provide an opening for similar community 
organizations to acquire and use state employment and wage data. Together, NYC Opportunity and 
CCF invested in a two-year effort (2016-2018) to facilitate the four community development corpo- 
rations’ access to state employment and wage data. 


Drawing from that effort, this report is intended as a guide for New York municipalities and com- 
munity organizations that may be considering seeking access to state UI wage data. It illustrates 
some challenges and opportunities and offers practical lessons. The lessons are specific to New York 
State, as other states have their own laws and processes for gaining access to state employment and 
wage data. Yet discussions about data security, data requests and analyses, and the usefulness of UI 
wage data may be relevant to all community organizations and their potential partners in this work. 


1 For more information, see “Data Sharing Under Labor Law §537” and “Data Sharing FAQs” on the 
NYSDOL website. 


2 To learn more about the New York City Change Capital Fund, read MDRC’s final report on its four-year 
economic mobility initiative (De La Rosa Aceves 2018). 


3 Building data capacity included strengthening the organizations’ management information systems and 
hiring or retaining staff members with data and evaluation capabilities. For more information, see De La 
Rosa Aceves, Greenberg, and Schell (2016). 
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BOX 1 


The New York State Law That Made This Effort Possible 


In 2013, the New York State Legislature passed the Unemployment Insurance Data Sharing Bill 
(S5773A). The bill amended NYS Labor Law §537 to expand access to unemployment insurance 
(Ul) wage data for certain entities for authorized purposes. According to the New York State 
Department of Labor: 


Government agencies, including the State University of New York (SUNY) and the City 
University of New York (CUNY), or agents or contractors of these agencies, can now receive 
and use UI data for: 


1. evaluating program effectiveness, including longitudinal outcome analysis; 

2. financial or other analysis required by federal, state or local law or regulation; 

3. preparation of reports required by federal, state or local law or regulation; 

4. improving the quality or delivery of program services or to create operational efficiencies; 


5. establishment of common case management systems between federal, state or local 
agencies delivering or supporting workforce services. 


SOURCE: New York State Department of Labor, Division of Research and Statistics, “Data Sharing Under 
Labor Law §537,” slide 3. 


This report describes the effort of CCF, NYC Opportunity, and MDRC to facilitate community 
organizations’ access to participant wage data and addresses the following questions: 


e¢ What is the New York State Labor Law amendment and what opportunities does it provide for 
expanded data access? 


¢ Under what conditions can organizations secure access to UI wage data? 
e What are some issues and challenges involved in accessing identifiable UI wage data? 


¢ What are some alternative ways of getting useful wage data if identifiable data cannot be obtained? 


Background on the Change Capital Fund and Its 
Investment in the Wage Data Study 


Since 2014, CCF, a collaborative of 15 funders, including NYC Opportunity, has been investing in 
community development corporations to increase economic mobility in persistently low-income neigh- 
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borhoods.‘ The four organizations participating in the first four-year cycle of CCF’s economic mobility 
initiative were Cypress Hills Local Development Corporation (CHLDC), Fifth Avenue Committee, 
New Settlement Apartments, and St. Nicks Alliance. These organizations offer a combination of 
services for local residents, including housing, education, and community-led planning. All four 
organizations offer workforce development services, and all track job placement and retention data. 


Like other workforce programs, these organizations found it challenging to stay in contact with 
former participants in order to track their employment status, earnings, and professional advance- 
ments over time. Access to more sources of administrative records, such as state employment and 
wage data, could help programs track outcomes more completely. 


The 2013 changes in the New York State Labor Law presented a chance for NYC Opportunity and 
CCF to explore whether the grantees could access and analyze state UI data. They selected MDRC 
through a competitive process to help the four community development corporations gain access 
to these data. MDRC was already conducting a formative evaluation of CCF’s initiative (2014-2018), 
providing ongoing advice that included extensive discussions with the organizations about their 
strategies to build data capacity and improve service delivery.® 


MDEC served as both the data intermediary and the technical assistance provider for this demon- 
stration because of its deep expertise in working with community organizations and advising on 
issues of consent and data security, and its extensive experience with NYSDOL employment and 
wage data requests.® As the data intermediary between the community development corporations 
and NYSDOL, MDEFC entered into a three-party agreement with NYC Opportunity and NYSDOL to 
ensure the secure transfer of data between NYSDOL and the grantees. MDRC also provided techni- 
cal assistance to the community organizations as they developed analysis questions, prepared data 
requests, and interpreted the data analyses. 


What the New York State Labor Law Amendment 
Means for Expanded Access to Ul Data 


It is difficult to assess how former program participants are faring in the labor market, because 
organizations have to rely on participants’ willingness to report and verify their job placements 
and earnings after they leave. If organizations had access to UI wage data, they could request com- 


4 NYC Opportunity initially worked as an adviser to the donor collaborative and later joined the fund as a 
donor. The donor collaborative and NYC Opportunity jointly funded the effort to access New York State Ul 
wage data. 

5 MDRC published a five-part series of briefs as well as a final report sharing lessons from CCF’s economic 
mobility initiative. 

6 MDRC has previously requested employment and wage data for its own research purposes. This effort 
was the first time that MDRC tried to facilitate community groups’ direct access to identifiable, individual- 
level wage and employment data. Other research organizations with similar qualifications could play the 
data intermediary role and provide technical assistance to community groups. 


How Community-Based Organizations Can Use New York State Employment and Wage Data 3 


prehensive, longitudinal data that would allow them to look at individuals’ earnings not only after 
but before they participated in the program. Understanding the earnings trajectories of program 
participants may help community-based organizations develop clearer, data-driven insights about 
the characteristics associated with favorable and less favorable outcomes. This could help the orga- 
nizations strengthen their programs for current and future participants. 


As summarized in Box 1, the 2013 labor law amendment provides an opportunity for certain or- 
ganizations to gain access to identifiable wage data from NYSDOL without an explicit data release 
authorization that has been signed by an individual and notarized. Data access is limited to “federal, 
state and local agencies, and their agents and contractors” for specific purposes, and only if federal 
and state confidentiality requirements are met. Organizations must apply for access to these data, 
and if the application is approved, they must pay to receive the data requested. 


For organizations operating workforce development programs, collecting authorizations from new 
or existing participants for release of UI wage data is time consuming. All the CCF grantees involved 
with this demonstration already had general release-of-information forms for incoming participants, 
but those releases were not compliant with the NYSDOL’s written authorization requirements.’ The 
amendment provided an opportunity to see whether the grantees, working in partnership with CCF, 
NYC Opportunity, and MDRC, could gain access to identifiable wage data in a potentially efficient 
and cost-saving way, and whether access to these data could facilitate the organizations’ efforts to 
use data to inform and improve their programs. 


What UI Wage Data Covers and Options for Data Access 


According to NYSDOL:8 
Each quarter, employers covered by New York State’s Unemployment Insurance Law are 
required to submit quarterly earnings information for each of their employees to the New 
York State Department of Taxation and Finance. This information is shared with NYSDOL 
to administer the state’s unemployment insurance program.? 


Although most nonfarm employment among New York State residents (about 97 percent) is covered 
by UI data, independent contracting jobs, informal jobs, and out-of-state jobs would not show up in 
the UI records.’® This means that while UI data can provide accurate and reliable employment and 


7 These requirements include specifically identifying which information will be disclosed, listing NYSDOL as 
the data source, and enumerating the purposes of collecting the data. For more information, see New York 
State Labor Law §537. 


8 This guide is nota NYSDOL publication. NYSDOL encourages any entity interested in obtaining data from 
NYSDOL to thoroughly review NYSDOL’s data sharing website (www.labor.ny.gov/data-sharing/). 


9 New York State Department of Labor (n.d.), p. 3. 


10 New York State Department of Labor (n.d.). For example, individuals employed in the gig economy, such as 
Uber and Lyft drivers, are considered independent contractors and not covered by the UI law. This has the 
potential to change based on a recent court ruling in New York that granted three Uber drivers the right to 
claim unemployment insurance. See Burns (2018). Other categories of employees not covered under the 
law include “some agricultural workers, railroad workers, private household workers, student workers, the 
self-employed, and unpaid family workers” (New York State Department of Labor, n.d., p. 3). 
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earnings information for the vast majority of individuals, they can result in some undercounting of 
those measures; the extent of undercounting can vary by occupation, employer type, and employer 
size, among other factors." It is important to consider how a lack of UI coverage might affect each 
group’s employment rate to avoid coming to the wrong conclusions when comparing groups. 


Wage information is available per quarter, although it does not include information on whether jobs 
are full time, part time, or seasonal. Information about the industry in which an individual is employed 
and an employer’s specific identification code may also be available.’? The data are matched with 
an individual’s Social Security number and are available from the year 2000 to the present.’ There 
is typically a three- to six-month time lag before the most recent and complete data are available. 


Under the 2013 law, NYSDOL can provide identifiable, or confidential, data without a program par- 
ticipant’s consent if legal and data security requirements are met. Data are considered identifiable if 
any of the wage records can be tied to a specific individual, either because those records are linked 
with personally identifiable information (such as Social Security numbers, names, or birth dates), 
or because a combination of data elements is enough to identify an individual. In most cases, unless 
the government agency has the internal data capacity to securely transfer and analyze identifiable 
program and UI data, a data intermediary must be engaged. 


As an alternative, organizations may decide to collect signed and notarized data release authorizations 
from participants. They can access identifiable UI data for anyone from whom they have obtained 
a written authorization. Organizations that decide to do this should work directly with NYSDOL to 
make sure their documentation of the authorization meets all the agency’s requirements. This pro- 
cess may be time consuming for a variety of reasons. An organization may not have a notary public 
available and on site to witness participant signatures. And if participants have left the program, 
the organization faces the same challenge of tracking them down regardless of whether it is trying 
to obtain an authorization or directly confirm job placement. 


The data can also be shared in deidentified and aggregate forms, which are not confidential and 
are therefore less complicated to obtain. Deidentified data are individual-level UI wage data with 
no personally identifiable information that are linked to a limited number of demographic or par- 
ticipation variables. In other words, the data are available on a person-by-person basis, but it is not 
possible to tell who each person is. Aggregate data are data summaries of groups of participants, 
groups that are large enough that the information cannot be used to identify anyone. Box 2 offers 
more details about the advantages and disadvantages of requesting identifiable data versus deidenti- 
fied or aggregate data. 


11 For example, job training programs that prepare workers for primarily Ul-covered jobs (such as commercial 
driver licensing programs) may appear to have graduates with higher employment rates than programs 
geared toward other sectors in which self-employment is more prevalent (Such as child care-related 
occupations), even though in reality the participants’ employment rates may be comparable. 


12 An employer identification code could allow community organizations to learn whether relationships with 
certain employers are particularly fruitful in terms of providing employment. Industry information may help 
community organizations identify whether their sector-based employment training connects participants 
with employment in a relevant industry. Ul records do not include occupation data. 


13 New York State Department of Labor (n.d.). 
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BOX 2 


Factors to Consider When Determining What Level of Data 
Would Benefit Your Practice 


Identifiable data 


e Quarterly wage data can be obtained and 
linked to an individual’s Social Security 
number. 


e The data request is processed more quickly 
at NYSDOL than the request for deidentified 
and aggregate data, since no data masking is 
required. 


e After data are received, analysis options can 
be added. 


e The administrative process for gaining access 
to the data can be lengthy. 


Deidentified or aggregate data 


e The administrative process is much shorter 
than that for identifiable, or confidential, data 
requests. 


e The data request takes longer for NYSDOL to 
process than the request for identifiable data 
and is more expensive. 


e No additional analysis, beyond what was 
prespecified to inform the data request, can 
be performed without submitting a new data 
request. 


e There is a risk of not receiving data for groups 
that are unlikely to be employed in Ul-covered 
jobs, because if only a few individuals have 
employment records the data could be 
identifiable. 


NOTES: A deidentified data file is an individual-level data file that is linked to selected variables of interest 
but cannot be linked to any specific individual. An aggregate data file is a data file that includes summary 


data for prespecified groups of individuals. 


Community organizations thinking about requesting identifiable, individual-level data may need to work with 
a data intermediary that has the infrastructure and capacity to meet NYSDOL's data security requirements. 
Staff members who will have access to individual-level data must complete NYSDOL’s Unemployment 


Insurance Confidentiality Training. 


Community organizations can use administrative UI wage records to measure employment outcomes, 
inform program improvement, and attract additional funding. Since state UI data include data from 
before and after program enrollment and completion, they can provide additional information about 
participants’ employment trajectories that can facilitate program monitoring and improvement 
efforts. Identifiable data can be used for diagnostic purposes at the participant or client level (for 
example, whether “Sandy Jones” might benefit from a specific job training program) as well as for 
many different analyses, while deidentified or aggregate data are limited to providing insights on 
how participants are doing as a group — for example, what proportion of graduates of an IT train- 
ing program are working and how much they are earning, or how many people like “Sandy Jones” 


are working. 
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The Demonstration 


Besides meeting the minimum requirements to gain access to identifiable, individual-level data, any 
proposal must be approved by the NYSDOL commissioner. To accomplish this, the demonstration 
involved a parallel process of discussions with NYSDOL and technical assistance sessions with the 
community organizations over an 18-month period. 


Working with the New York State Department of Labor 


MDRC, NYC Opportunity, and CCF, in consultation with the community development corpora- 
tions, spent over a year developing an agreement with NYSDOL to facilitate the groups’ access to 
state UI wage data."4 The original proposal called for MDRC to provide technical assistance to 
program staff members at the four community development corporations as they developed analy- 
sis questions, conducted analyses, and interpreted their findings. NYSDOL reviewed and rejected 


that first proposal as well as two more (see Box 3)."® 


Ultimately, the fourth proposal was accepted 
and an agreement executed.'® MDRC submitted the program participants’ Social Security numbers 
(provided by the grantees) with a request for UI wage data in mid-February 2018 and received the 


data later that month. 


Ultimately, NYSDOL did not grant the community organizations access to confidential UI data 
because they determined that the organizations did not have a direct contractual relationship with 
the municipal government and so did not qualify for access. Consequently, MDRC restructured its 
work with the organizations and assumed data analysis responsibilities. NYSDOL then required 
NYC Opportunity to submit documentation detailing MDRC’s new data analysis responsibilities 
before executing an agreement for the UI wage data request." 


Lessons Learned from Discussions with NYSDOL 

The contract development process offers insights for other municipalities and community organiza- 
tions that might be considering applying for access to UI wage data. Through this iterative process 
with NYSDOL, a few important lessons emerged: 


14 Discussions began in January 2017 and were completed in February 2018. 


15 All proposals had MDRC acting as the data intermediary supporting the secure transfer of Social Security 
numbers from the community groups to NYSDOL, rather than the community organizations entering into a 
direct agreement with the state. 


16 Before any proposal can be approved by NYSDOL and an agreement executed, a nondisclosure 
acknowledgment must be signed by each person who will work with the data and submitted to NYSDOL. 


17 NYC Opportunity was quick to revise a preexisting task order, detailing the roles of the community 
organizations and MDRC, so that NYSDOL could execute a data agreement. The three-party agreement 
between NYSDOL, NYC Opportunity, and MDRC specified that no one other than the MDPC staff 
members listed therein would have access to the identifiable data. 
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e Engaging in regular communication about the details of a data sharing proposal is crucial to 


The legal teams at both NYSDOL and MDRC worked hard to understand the demonstration’s goals 
and data needs, recognizing the importance of balancing data access with protection of individuals’ 
identities. Because the law was new, and granting data access to direct service organization staff 
members under these particular rules had not been attempted in New York, the effort involved an 


BOX 3 
Submitted Proposals 


REJECTED 


Proposal 1: MDRC would transfer program data from the community organizations to NYSDOL. 
The community organizations would then upload the requested wage data to their respective 
management information systems (MISs) and conduct analysis through those systems. The 
MISs already met NYSDOL's encryption standards and could isolate which staff members were 
able to see the data. Only staff members who received NYSDOL training would have access to 
the identifiable, individual-level data. MDRC would provide guidance throughout the entire data 
transfer and analysis process, including the interpretation of the analysis. 


Proposal 2: MDRC would transfer program data from the community organizations to NYSDOL. 
The community organizations would use MDRC’s virtual desktop interface to conduct all wage 
data analysis — the data would never be uploaded to any community organization’s MIS, and only 
staff members who received NYSDOL training would have access to the identifiable, individual- 
level data. Organization staff members could share the aggregate data analysis with other staff 
members. MDRC would provide technical assistance with data aggregation and the interpretation 
of the data analysis. 


Proposal 3: MDRC would submit the data request to NYSDOL on behalf of the community 
organizations. Staff members from the community organizations would come to MDRC’s offices 
to conduct analysis of the wage data and their own program data and could carry only aggregated 
data off site. MDRC would provide on-site technical assistance, including assistance with the 
interpretation of the data analysis. 


ACCEPTED 


Proposal 4: The community organizations would send MDRC the program data relevant to their 
analysis questions. MDRC would request identifiable, individual-level employment and wage data 
from NYSDOL and conduct analysis on behalf of the organizations. MDRC staff members working 
with the data would complete the NYSDOL training. MDRC could share aggregate-level analysis 
with the community organizations and provide technical assistance with the interpretation of the 
data analysis. 


reaching a workable agreement. 


iterative process of developing proposals that took much longer than anticipated. 
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e The contract structure is important. 


Since the community organizations, as nongovernmental agencies, would not by themselves qualify to 
gain access to confidential UI data from NYSDOL, they needed to be in a direct contractual relation- 
ship with an agency (in this case, a New York City agency) to qualify to receive the data as “agents or 
contractors of a government agency.” It was not enough for them to be grantees of CCF, which had a 
contractual relationship with NYC Opportunity. NYSDOL and MDRC discussed the possibility of 
allowing one staff member from each grantee to access the data, with NYC Opportunity and MDRC 
taking on full responsibility for the security and confidentiality of the specified individuals in each 
organization. Ultimately, NYSDOL decided that this arrangement was not compliant with the new law. 


¢ Data security concerns are paramount. 


Even with the correct contract structure, NYSDOL has strict security requirements for the storage 
and transfer of confidential UI data, and many organizations do not have the infrastructure to meet 
these standards. Because some of the CCF grantees did not have this capacity, MDRC provided the 
secure data infrastructure and handled the data transfers on behalf of the organizations. 


Working with Community-Based Organizations 


While discussions about the data sharing contract were occurring with NYSDOL, MDRC was also 
helping the CCF grantees prepare for the expanded wage data access. The demonstration launched 
in December 2016 when representatives from MDRC, CCF, and NYC Opportunity met separately 
with each of the four community development corporations to discuss their workforce programs and 
interest in learning from the wage data. These conversations identified which workforce programs 
collected the most complete information about current and past participants, including Social Security 
numbers, demographics, and information on program enrollment, participation, and completion."® 
After these assessments, MDRC helped the organizations develop analysis questions that could be 
answered by linking their program data with the UI data. This process helped MDRC understand 
the potential limitations of analyzing UI data. 


The four community development corporations were recipients of a four-year (2014-2018) invest- 
ment to build their data capacity, strengthen management information system (MIS) capabilities, 
and hire or retain staff members with data and evaluation capabilities. Their staff members were 
excited by the possibility of gaining access to identifiable UI data that could provide insights about 
the employment and wages of participants who completed their programs, or that could inform 
customized outreach plans for individual participants. Additionally, they hoped to use the data to 
report their program outcomes to current and potential funders. In particular, they were interested 
in data on postprogram wage outcomes to see whether participants who completed their programs 
had different employment experiences from those who did not. They wanted to maintain flexibility 
in their analysis plans, in case other questions occurred to them after they started working with the 


18 MDRC also inquired about how long each program had been operating, whether the program served 
participants in cohorts or had ongoing enrollment, and the total number of participants served in a given 
year. 
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data. They were eventually disappointed that they would not be granted direct access to the identifi- 
able UI data but were still interested in seeing the aggregate results. 


Each program designated a point of contact who served as the liaison between MDRC and the or- 
ganization’s program staff, most directly shaped analysis questions, and prepared program data for 
submission to NYSDOL. MDRC worked with the community development corporations over 12 
months to help them refine their analysis questions, review their management information systems, 
confirm participant consent processes,'® and prepare data request files to submit to NYSDOL.”° Box 
4 presents some examples of analysis questions proposed by the organizations. 


BOX 4 

Sample Analysis Questions 
Among participants in each program, how do employment and earnings trajectories differ between 
the following groups? 
e Participants who receive services from multiple partners and participants in only one program 
e Participants who completed the program and those who did not complete 
e Participants who were placed in jobs and those who were not placed 
e Participants who were enrolled in a program and those who were not enrolled 


e Participants grouped by demographic characteristics (gender, race/ethnicity, parent 
demographics, degree completion) 


e Different cohorts of participants (groups of participants who enrolled in the same program at the 
same time) 


Among participants in each skills training program: 
e Do they find employment in the expected industries? 


e Do they retain employment in the expected industries? 


MDRC completed selected analyses of the UI data that had been requested by the community 
organizations and shared them with each organization’s point person during a one-day technical 
assistance convening in April 2018 at MDRC’s offices. The purpose of the convening was to help 
the data staff at the organizations understand the employment and earnings measures, interpret 


19 Obtaining consent is not a requirement of the law, but MDRC considers this good practice. 


20 A data request file contains the Social Security numbers for participants for whom the site would like wage 
and employment data. Other demographic and program participation indicators were provided to MDRC 
and linked to each individual. Before data can be transferred, data transfer agreements must be put into 
place between the data intermediary and the community group. 
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the analyses, and explore additional queries that, given more time, might be possible to investigate 
with UI data. Staff members from each organization then developed one or two follow-up analysis 
questions, and MDRC conducted and shared additional analyses with each organization in June 2018 
in a guidance document to help the organizations understand the outcomes. In early 2019, MDRC 
followed up with each organization to discuss how the demonstration had gone and capture staff 
perspectives on both the usefulness and the limitations of the UI data. 


CCF Grantees’ Perspectives on the Potential Value of Access to Deidentified Wage Data 
MDRC’s conversations with the grantees uncovered some lessons learned from the demonstration: 


¢ The completeness and quality of participant data in the organizations’ management information 
systems determine how useful UI data can be, regardless of whether the data are identifiable. 


Three of the grantees indicated that they are now more conscious and thorough about collecting 
key identifying and demographic information from new participants during intake. Preparing data 
requests for MDRC and seeing the results of the data analysis helped staff members see the impor- 
tance of collecting accurate Social Security numbers, recording enrollment and participation dates 
accurately, and tracking important indicators (such as educational attainment or completion status) 
comprehensively, in order to understand employment trends across different groups of participants. 
(The one grantee that reported no change in data collection or management practices during or 
after the wage data study already tracked and recorded data for its participants; its staff credited its 
comprehensive database in part to its participation in a different recent MDRC evaluation.) 


¢ Efforts to use UI data are more efficient if the data management is more centralized. 


One grantee wanted to use UI data to assess how participants who were engaged with multiple enti- 
ties in a partnership were faring in the labor market compared with participants who were engaged 
with only one entity. Because the partners had different systems and collected different types of 
information on their participants, and because this was a relatively new partnership initiative, it 
was difficult to compile a sample with complete information on participants who had been engaged 
with the grantee long enough to have two full years of postenrollment UI data. 


Two of the grantees were interested in collecting data from participants whose data were tracked 
in more than one database across the same organization. This is typical of organizations that have 
more than one funding source to support their workforce programming, as funders often require 
their grantees to use proprietary databases. It took some time to extract appropriate data from dif- 
ferent databases and make sure the common elements were consistently recorded. One organization 
had one central database that tracked all participants in their workforce programs, so it was easy 
for them to extract the data they needed for the demonstration. 


¢ Identifiable data are preferred over deidentified or aggregate data, but aggregate data are still 
informative and valuable. 


All the grantees would have preferred to receive identifiable earnings data that they could merge 
with their existing demographic and participation data so that they could use the combination to 
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improve services for specific individuals and have the flexibility for additional analyses as other 
questions arose. Although all grantees gained some insight from the aggregate data, their impressions 
of the data’s usefulness varied quite a bit. One organization found them minimally useful, because 
staff members primarily wanted to find out whether specific former participants they were no lon- 
ger able to track were employed. Two organizations found these data moderately useful, indicating 
that they gained valuable information from the analysis but expressing frustration that the sample 
size limitations of aggregate data did not allow them to fully answer their analysis questions. The 
fourth organization acknowledged the advantages of identifiable data but saw many uses for the 
aggregate results. The staff discussed ideas such as talking to individual participants about average 
postprogram outcomes, using subgroup analyses for program improvement, and communicating 
with funders about their participants’ employment trends. 


¢ Organizations would find it more valuable to receive regular updates to the data than a one- 
time data transfer. 


Since the iterative process with NYSDOL on formalizing a data sharing agreement took more than a 
year, MDRC had time to make only one data request to NYSDOL for each grantee. All the grantees 
would have liked an opportunity to request more data from NYSDOL so that they could refine their 
requests, request data for more people (for whom they now had more complete information), and see 
changes over time. Two of the grantees expressed an interest in prespecifying aggregate measures 
that they could request yearly, recognizing limitations in their staff capacity and the need for ad- 
ditional technical assistance to help them analyze or interpret data that would be either identifiable 
or sent to them more frequently. 


¢ Short-term technical assistance from a data intermediary can build staff capabilities for using 
and interpreting data. 


MDEC provided direct, ongoing technical assistance with developing analysis questions, assessing 
systems used to track participant data, preparing data request files, and interpreting results with 
the points of contact at each organization from December 2016 through June 2018. Staff members 
indicated that the assistance with developing questions led to improved data collection and tracking, 
once they realized the importance of having good intake and participation data. Technical assistance 
on some aspects of data interpretation may be more long term. While all staff members grasped 
how to use wage data to follow a group’s earnings trajectories from one year to the next, additional 
assistance was needed to explain that comparing two different groups of participants may not be a 
good indicator of whether a program was effective. For example, seeing higher employment rates 
among program completers than among program dropouts may reflect different motivation levels 
among the participants rather than program effectiveness. 


¢ Fostering a more data-driven perspective across the organization’s program and management 
staff can make the data more useful. 


Organizations that engaged both the data point of contact and other nontechnical staff members 


in this project had more robust and realistic plans for UI wage data than organizations that relied 
primarily on the data point of contact to drive the analysis plans. As an illustration, multiple staff 
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members at each community organization saw the guidance document that MDRC provided to the 
grantees in June 2018 and were involved in the final conversations with MDRC. There was a mix of 
engagement during each conversation; some discussions reflected the perspective of the primary 
point of contact for the wage data study (that is, the point of contact did most of the sharing), while 
others reflected a broader mix of perspectives (staff members spent equal time sharing their experi- 
ences). The discussions that involved a mix of perspectives demonstrated a broader understanding of 
the data results and more thoughtful ideas on how they could use these data to talk to participants, 
improve program pathways, and communicate with potential funders. 


Appendix A describes some specific insights that Cypress Hills Local Development Corporation 
(CHLDC) gained through the demonstration. CHLDC, which hired a full-time evaluation director 
with CCF funds, requested UI data for over 3,000 participants across five of its programs for whom 
it had the most complete relevant data. Staff members were especially thoughtful about the limita- 
tions of aggregate data, and this helped them to articulate and prespecify questions that could be 
answered within those limitations. When MDRC shared the aggregate findings, CHLDC made sure 
staff members with different roles across the organization understood the findings and could talk 
about them accurately. 


Recommendations to City Agencies and 
Community-Based Organizations 


The investment in the wage data demonstration was a valuable experience for the CCF grantees, as 
well as for NYC Opportunity, CCF, and MDRC. Although the effort did not result in the community 
organizations getting access to identifiable UI data, it did allow them to obtain aggregate UI data on 
their current and former participants that would not have been possible before 2013, and it offered 
useful lessons on the processes of working with NYSDOL and the organizations. 


Considerations for Organizations Interested in Accessing UI Wage Data 


As described earlier, NYSDOL can provide UI data to organizations in different ways. Identifiable 
UI data are available but call for significant investments in staff and infrastructure to make sure that 
security and confidentiality requirements are met. Once these standards are in place, identifiable 
data offer the most flexibility in terms of analysis and can inform service improvement for specific 
individuals, and follow-up data files are relatively cheap to request. Deidentified or aggregate data 
are not confidential and therefore do not need to be secured, so gaining access to these files is easier. 
However, the staff labor required to prepare the requests for aggregate data and for NYSDOL to pro- 
cess these files for delivery makes these requests more time intensive, and therefore more expensive. 


START PLANNING EARLY. Figure 1 illustrates a suggested planning process and checklist for com- 
munity organizations that are interested in accessing UI wage data from NYSDOL. Data sharing 
agreements can take a long time to develop and formalize, so organizations should start planning 
for data acquisition at least a year before they hope to gain access to the data. The first step is to 
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FIGURE 1 


Process Checklist for Agencies Considering Seeking Access to 
New York State Department of Labor (NYSDOL) Wage Data 


START 
MONTH 10 


py-vel(e(-Wimr-Vare| 
Md aYel ae Fe) a] 


would be 


Prepare for 
discussions 
with NYSDOL 


(@) oh Via i-)a) 
executed 
agreement 


Process and 
use the data 


from NYSDOL 


ESTIMATED TIME: 2 MONTHS ESTIMATED TIME: ESTIMATED TIME: 
2 MONTHS | 4 MONTHS 2 MONTHS 


Note: Begin this process as early as possible, at least 12 months before you need access to the data. 
Timing can fluctuate based on the data requested. 


helpful 
P ESTIMATED TIME: 


C]Hold a meeting with the community organization to determine whether access to UI data 
would be helpful to the organization. Identify specific ways the data would be helpful and ask 
NYSDOL whether community groups will be able to access and use data for this purpose. 


C]ldentify which programs might benefit most from accessing UI wage data, and determine 
whether these programs collect Social Security numbers from participants. Social Security 
numbers are necessary for any request for wage data records from NYSDOL. 


(]Decide whether the community organization should access identifiable, individual-level data or 
deidentified or aggregate data, and ask NYSDOL about special considerations for accessing 
these types of data. 


(] Determine whether the community organization has the staff capacity to send data securely to 
NYSDOL, specify data requests, and conduct data analysis. If it does not have the capacity, 
identify a data intermediary and technical assistance provider to help the organization. 


CJIf using a data intermediary to access data on behalf of the organization: 


+ Establish a task order between a federal, state, or local government agency and the data 
intermediary. 


+ Establish a contractual relationship between the agency and the community organization 
specific to the Ul wage data request. 


+ Establish points of contact at the agency, community organization, and data intermediary to 
streamline communication throughout the negotiation process with NYSDOL. 
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ACCESS NYS 
WAGE AND 
EMPLOY- 
MENT DATA 


(continued) 


TASK 
4 


FIGURE 1 (continued) 


[]Complete the NYSDOL application to request UI data. To do this, you will need the following 
information: 


« Names of the government agency and community organization that are working together 
- Type of data to be requested (identifiable vs. deidentified or aggregate) 
- Description of how NYSDOL data security requirements will be met 


CJEstablish a data sharing agreement between the data intermediary and the community organi- 
zation if a data intermediary is involved. 


C]The organization receiving the data, as well as individual staff members working with the data, 
will need to complete nondisclosure acknowledgments and submit them to NYSDOL before a 
data agreement can be executed. All staff members with access to UI data will also need to 
complete NYSDOL data security training before the agreement can be finalized. 


TIP if a data intermediary is involved, the intermediary should lead the 


discussions with NYSDOL. 


(_JAfter the application is submitted to NYSDOL, anticipate some back and forth to clarify any 
questions about the application. 


(JOnce NYSDOL decides that it has sufficient information about the data request, it will generate 
a data agreement for signature. 


(JAs soon as NYSDOL executes the data agreement, UI data can be requested. 


(JProcess, analyze, and interpret the wage data. The community organization may need to 
engage a technical assistance provider if it does not have internal staff capacity to perform 
these activities. 


TIP A data intermediary can provide support and technical assistance with 
checking the data, running the analyses, and interpreting the findings. 
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determine whether UI wage data would be useful and, if so, what kind of data. For example, college 
access programs will probably get more useful information if they request data on former partici- 
pants after graduation, rather than on current participants who are in school and less likely to be 
employed. On the other hand, training or certificate programs might find it useful to request UI 
wage data that cover the periods both before and after participants receive credentials to see whether 
more participants found jobs, or whether they switched to higher-paying or more stable jobs. The 
organization then needs to determine whether its information on the groups for whom it seeks wage 
data is sufficiently complete to take advantage of the administrative data. NYSDOL requires matches 
on Social Security numbers, so if these are missing or invalid for a large proportion of participants, 
requesting wage data may not be worth the effort. 


PREPARATION IS KEY TO BENEFITING FROM THE DATA ACCESS. Because the wage data dem- 
onstration was time limited and the agreement with NYSDOL took much longer to finalize than 
anticipated, the CCF grantees were not able to benefit from all that the UI wage data requests could 
have offered. They all agreed that the UI data and selected analyses were informative and valuable, 
and they had adjusted requests in mind in case another opportunity to request data arose in the 
future. The general consensus among the grantees was that organizations should make sure their 
participants’ Social Security numbers and key demographic and participation data are complete 
and accurate, that staff members have the time and capacity to make good use of the data, and that 
analysis questions are thoughtfully prioritized before investing in the process. 


Deciding Between Identifiable UI Data and Deidentified or Aggregate Data 


CONSIDER THE FEASIBILITY OF OBTAINING AUTHORIZATIONS. An organization can try to access 
identifiable UI data by demonstrating that it has obtained written data-release authorizations from 
all participants for whom it is interested in collecting the data. This effort could represent a huge 
undertaking that may not be practical, especially with past participants who are no longer engaged 
with the organization. It might make sense, however, for the organization to begin collecting autho- 
rizations that have been cleared by NYSDOL for current and incoming participants, particularly if 
it is starting a new program for which UI data would be useful in the future. 


CONSIDER WHETHER THE ORGANIZATION CAN MEET THE REQUIREMENTS FOR IDENTIFIABLE 
DATA. Lacking state-approved authorizations, an organization can try to access data under the new 
labor law. The organization should discuss conditions under which it might qualify for these data 
with its federal, state, or local agency partners and NYSDOL, particularly when it is necessary to 
engage a data intermediary to assist with data security requirements. Although this effort did not 
result in access to identifiable data for the community organizations, it did confirm that a direct 
contractual relationship needs to exist between the organization and a government partner. An 
organization with a qualified contractual relationship should plan for a three-month response time 
after the partner agency first submits an application for access to NYSDOL data?" and be prepared 


21 The three-month period accounts for a 20-business-day NYSDOL review that may result in a request for 
more information; 20 business days for the qualified agency to provide additional information; and 30 
calendar days for NYSDOL to provide an approval or denial of application. See the FAQs on NYSDOL’s 
data sharing website. 
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with a backup plan to request deidentified or aggregate data if NYSDOL does not approve the pro- 
posal. It should also weigh the opportunity costs of pursuing this option before investing the effort 
up front: Staff members at community organizations are often already stretched to capacity, and the 
organization should think about what current work they might have to delay or give up in order to 
spend time with the data. Even if NYSDOL ultimately approves the request for confidential UI data, 
the organization will need internal staff capacity to process the data and perform the analysis. This 
option may make more sense for organizations that have staff members dedicated to research and 
data analysis, or that can work with a data intermediary or technical assistance provider, than for 
an organization whose data staff also has substantial direct service responsibilities. 


PLAN CAREFULLY TO MAXIMIZE THE VALUE OF DEIDENTIFIED OR AGGREGATE DATA. Requesting 
deidentified or aggregate data is much easier from a legal perspective, since these data are not sub- 
ject to the same data security requirements as confidential UI data. Organizations will still need 
to dedicate considerable staff time to formulating the data requests. For deidentified data files, the 
staff will need to decide which groupings of participants are the highest priority. If staff members 
request data on too many overlapping groups, the data become identifiable and therefore confiden- 
tial, and organizations will need to cut down the number of requests.” For aggregate data files, the 
staff will also need to decide the level of aggregation. A highly skilled technical staff member or a 
data intermediary could help the organization figure out the minimum level of aggregation needed 
to maximize the potential of the analysis without requesting identifiable UI data. It is important to 
note, though, that more complicated aggregate data queries might require more time for NYSDOL 
to extract the data, which would result in more expensive data files. Requesting deidentified or ag- 
gregate data may defray some of the up-front setup costs that a request for identifiable UI data would 
require, but each request made to NYSDOL may be more costly to obtain.?* 


CONSIDER THE DEMANDS OF DATA INTERPRETATION. For organizations that cannot afford to 
have staff members spend a lot of time on data processing and analysis, it may make sense to invest 
some up-front planning time on prespecifying deidentified or aggregate requests that are easy to 
read and interpret. 


Potential Uses and Limitations of UI Wage Data for Community Organizations 


LONGITUDINAL DATA CAN HELP ORGANIZATIONS TRACK, COMPARE, TARGET, AND IMPROVE. 
Community-based organizations, specifically those that provide workforce development programs, 
have routinely reported program activity and participant job placement to their funders. Increasingly, 
funders are requesting that organizations move beyond this type of reporting and perform data 
analysis to assess program outcomes. Both funders and providers are interested in tracking the longer- 
term employment and earnings results of program participants. Longer-term results could show 


22 Asan example, an organization might be interested in looking at employment trajectories by gender, 
age, educational attainment, and enrollment cohort. Each group by itself would probably yield enough 
employment records to preserve confidentiality, but if there were, say, very few females under 24 with high 
school degrees in a particular cohort, the organization would have to decide which group indicator to drop 
from the file so that the participants could not be identified. 


23 Organizations should talk with NYSDOL about cost estimates for specific aggregate requests, since they 
can vary widely. 
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earnings trajectories for training program participants and allow for comparison across programs; 
identify the participant cohorts (that is, groups of people who participated in the same program at 
the same time) that had better earnings trajectories; or show which cohorts may need additional 
help or which programs may require a new strategy. Organizations could see whether participants 
who attended sector-based training courses ended up in the expected or different industries. And 
employment data on formerly incarcerated participants could help an organization identify and 
partner with employers who regularly hire them. In these ways, state employment and earnings data 
provide a promising resource for understanding program outcomes and the longer-term experiences 
of program participants. 


IDENTIFIABLE DATA HELP PROGRAMS TARGET ASSISTANCE. Community organizations can cer- 
tainly make their most informed decisions about enhancing program activities using identifiable 
data, assuming that they have the staff capacity to analyze them. Staff members who are able to 
access the identifiable data can take a “whole-person” view of the data to make informed judgments 
of what might be beneficial for a specific participant. For example, staff members might recommend 
different programs to participants without high school diplomas who are stably employed and those 
who are not employed. They could target their outreach to specific participants according to what 
they see in the data. 


IDENTIFIABLE DATA OFFER FLEXIBILITY IN ANALYSIS. Depending on the quality of their enrollment 
and participation data, organizations could also use individual-level data to look at how different 
subgroups of interest are doing. The advantage of having identifiable data for these purposes is that 
this analysis could be done as the need arises and does not need to be prespecified. For example, if 
the organization is deciding whether to target a particular subgroup (such as young adults who have 
been involved with the justice system) with a specific type of employment service, staff members 
could look at that subgroup’s historical employment data to assess the extent of the need without 
submitting an additional customized data request to NYSDOL. 


DEIDENTIFIED OR AGGREGATE DATA CAN GUIDE PROGRAM IMPROVEMENTS. If organizations 
request deidentified or aggregate data rather than identifiable UI data, they cannot assess needs 
or target interventions for specific people, nor can they conduct analysis on groups that were not 
prespecified. However, deidentified or aggregate data can be useful for tracking participant employ- 
ment or earnings changes for key subgroups or programs. Organizations that run sectoral training 
programs, for example, could request aggregate UI data by program cohort to see whether their 
sectoral training participants are finding or keeping employment in related industries. Or organi- 
zations could examine earnings data to assess whether participants without high school diplomas 
need different kinds of employment services than do more educated participants. 


AGGREGATE WAGE DATA CAN ALSO BE USEFUL FOR TRACKING PERFORMANCE METRICS. The 
organization would have to specify the aggregation for NYSDOL, grouping participants by program 
or demographic indicator. Because of the three- to six-month lag in UI data collection, the metrics 
may not be current, but they would reflect more accurate and complete information for program 
participants than self-reported information. Organizations could arrange for periodic requests and 
control costs by keeping the groups and metrics constant. 
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Conclusion 


The investment in the wage data demonstration was a tremendous opportunity for CCF grantees to 
obtain additional earnings information on their participants, use the information to address some 
questions they had about their programs, and think about how these data might contribute to future 
program improvement, performance monitoring, and fundraising activities. For the partners in the 
project — MDRC, CCF, and NYC Opportunity — this initial effort to help community organizations 
benefit from the 2013 New York State Labor Law amendment has provided a base of knowledge on 
which future efforts can build. 
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APPENDIX 


A 


Accessing and Using Ul Wage Data at 
Cypress Hills Local Development Corporation 


Staff members at Cypress Hills Local Development Corporation (CHLDC) were enthusiastic par- 
ticipants in the wage data demonstration. At the outset, they had realistic expectations, recognizing 
that while identifiable, individual-level data were preferable to aggregate data, using identifiable data 
would require the greatest time commitment from their staff. To maximize the usefulness of the state 
unemployment insurance (UI) data, staff members calculated participant counts for programs and 
subgroups to prioritize the groups that were most likely to yield enough employment records to draw 
certain conclusions. For example, they prioritized programs for which they had the most complete 
Social Security number information and invested substantial time in making sure those numbers 
were accurately entered in their data systems. They also identified sufficiently populous subgroups 
within their larger programs, including subgroups determined by preenrollment characteristics 
such as high school equivalency attainment and postenrollment outcomes such as credential receipt. 


Appendix Table A.1 presents selected employment and earnings outcomes for participants in three 
of CHLDC’s workforce programs. The Young Adult Internship Program (YAIP) is a 14-week paid 
internship for disconnected young adults between ages 17 and 24; CHAMPION Network is a 2-week 


APPENDIX TABLE A.1 


Average Employment and Earnings of Cypress Hills 
Local Development Corporation Participants, by Workforce Program 


CHAMPION BUILDING 
OUTCOME YAIP NETWORK WORKS 
Year before enrollment 
Ever employed (%) 36.2 53.6 64.8 
Average earnings ($) 2,375 4,197 7,494 
Average earnings, of those employed ($) 6,568 7,834 11,566 
Year 1 
Ever employed (%) 95.0 93.5 95.8 
Average earnings ($) 6,015 10,689 15,700 
Average earnings, of those employed ($) 6,332 11,438 16,392 
Year 2 
Ever employed (%) 77.3 82.1 91.5 
Average earnings ($) 7,282 12,104 28,303 
Average earnings, of those employed ($) 9,420 14,735 30,915 
Sample size 260 168 71 


SOURCES: MDRC calculations using wage records from the New York State Department of Labor (NYSDOL) 
and program records from Change Capital Fund grantees. 


NOTE: The table reflects data on participants who enrolled in the Young Adult Internship Program (YAIP), 
CHAMPION Network, or Building Works between August 2011 and September 2015. 
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training course for adults up to age 30 that ends with a commercial driving credential and immedi- 
ate job placement upon completion; and Building Works is a 14-week preapprenticeship training 
program that prepares adults (mostly younger than 30) for careers in construction. 


MDRC’s guidance document provided some limited insights: 


e About half the CHAMPION participants came with previous employment experience in the year 
before enrollment, compared with a third of YAIP participants and two-thirds of Building Works 
participants. 


e Almost all the participants in these three programs were placed in a job in Year 1. Of the three 
programs, Building Works participants had the highest earnings levels. (The smaller number of 
participants in Building Works means that participants with unusually high or low earnings can 
affect the average earnings more so than in larger programs.) 


e Employment levels for participants across all three programs dropped in Year 2, but average 
earnings rose. Building Works participants were more likely than participants in the other pro- 
grams to remain employed. They also averaged the highest earnings in the second year — almost 
doubling their earnings from Year 1. 


Follow-up conversations with CHLDC staff members uncovered more insights. Before seeing the UI 
data, CHLDC had been able to verify employment for about two-thirds of its participants, so the UI 
employment outcomes were encouraging. Staff members indicated that it took an average of three 
to four months after program enrollment for a participant to be placed in a job, which explains why 
a higher employment rate with lower earnings in Year 1 than in Year 2 would make sense. They 
also noted that the UI annual earnings for their CHAMPION participants seemed low, since the 
current average wage for CHAMPION job placements was around $16 per hour. But they recalled 
that in earlier years, CHAMPION had placed participants in much lower-paying jobs (with hourly 
wages ranging from $8 to $12) when the minimum wage in New York was still $7.25. Because the 
table reflects earnings for all participants with two full years of follow-up data, and not just the 
most recent participants, the lower average reflects those past wages, accounting for the apparent 
mismatch with their initial expectations. 


CHLDC and MDRC took away a few key points about their programs: 


¢ Participants across all three programs had substantially higher employment rates and earnings 
two years after enrolling in their respective programs at CHLDC. Participants were on average 
earning more than three times what they were earning before they came to CHLDC. For example, 
individuals earned about $4,000 in the year before they enrolled in CHAMPION and were earn- 
ing about $12,000 in Year 2. 


¢ Participants in Building Works seemed to do particularly well; two years after enrolling in the 


program, they were maintaining nearly full employment. Additional findings shared with CHLDC 
indicate that most of them were still working in construction-type jobs. 
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CHLDC intends to use these findings for a few purposes: 


¢ Internal planning. Staff members wanted to see a higher percentage of their CHAMPION par- 
ticipants end up in the industries for which they received training. They can use these numbers 
for internal goal-setting purposes. (Industry outcomes were shared with CHLDC but are not 
shown in this document.) 


¢ Motivating current and incoming participants. Staff members thought these data would be use- 
ful as a way to discuss expected outcomes with participants. Less engaged participants might be 
more likely to persist in earning certain credentials if they know that on average, past participants 
have seen higher earnings and better employment prospects. 


¢ Generating excitement among funders. Staff members hope to use these highlights in future 
conversations with funders and funding proposals. 
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